Effect of changing multiple choice questions from "all of the above" to "select all that apply".

INTRODUCTION: The purpose of this study was to describe the effect of converting multiple choice questions (MCQs) that include an "all of the above" (AOTA) answer option to a "select all that apply" (SATA) question type on question performance. METHODS: A summative assessment at the end of the first professional pharmacy year was comprised of approximately 50 multiple choice questions covering material from all courses taught. Eight questions contained AOTA answer options and were converted to SATA items in the subsequent year by eliminating the AOTA option and including the words "select all that apply" in the stem. Majority of the other questions included on the exam remained the same between the two years. Item difficulty, item discrimination, point biserial, and distractor efficiency were used to compare the MCQs on exams in the two years. RESULTS: The AOTA questions were significantly easier and less discriminating than the SATA items. The performance of the remaining questions on the exam did not differ between the years. The distractor efficiency increased significantly when the questions were converted to SATA items. CONCLUSIONS: MCQs with AOTA answer options are discouraged due to poor item construction resulting in poor discrimination between high and low performing students. The AOTA questions are easily converted to the SATA format. The result of this conversion is a more difficult and more discriminating question with all answer options chosen, which prevents students from easily guessing the correct answer.

Falling NAPLEX pass rates are cause for concern.

INTRODUCTION: First-time pass rates of the North American Pharmacist Licensure Examination (NAPLEX) have declined 7% from 2019 to 2022 with more than a third of schools experiencing a decline of ≥10%. COMMENTARY: The cause of the decline is likely multifactorial and extends beyond the impact of the COVID-19 pandemic. Changes to the NAPLEX blueprint in 2021, curricular revisions in response to the implementation of Accreditation Council for Pharmacy Education Standards 2016, and changes to prerequisite course requirements in response to declining enrollment must also be evaluated as potential causes. IMPLICATIONS: The academy must respond to this decline by scrutinizing admissions, curriculum, and assessment processes. We urge the National Association of Boards of Pharmacy to provide access to student-level data on NAPLEX performance and increase transparency in passing standard practices to inform this process.

Evolution of a capstone exam for third-year doctor of pharmacy students.

BACKGROUND AND PURPOSE: Pharmacy capstone exams are an example of an assessment strategy to evaluate student competency and practice-readiness. Capstone exam processes have been discussed in the literature; however, description of format, evolution of the process, and implementation in the year prior to rotations is lacking. This manuscript discusses the development, implementation, and utilization of a third professional year pharmacy capstone exam to assess advanced pharmacy practice experience (APPE) readiness. EDUCATIONAL ACTIVITY AND SETTING: A pharmacy year three (PY3) capstone exam was initially developed prior to enrollment of the inaugural graduating class. Since the first iteration, utility has remained the same, but the overall development and implementation process has evolved due to changes in programmatic and student learning outcomes, curricular mapping, student and faculty feedback, and practice expectations. FINDINGS: The entities responsible for exam implementation perceive faculty and students have a positive perception of the exam evolution and improvements implemented. Specifically, feedback from students and faculty mentioned clear expectations, comfort with the new capstone exam process, and an appreciation for the focus on interrater reliability. As assessment is an iterative process, quality improvement strategies continue to be implemented to address feedback regarding the PY3 capstone exam. SUMMARY: Purposeful utilization of capstone exams in the third year is one method to assess APPE readiness. Sharing one college's experience with the evolution of such an exam, quality improvement methods, and experience in implementing reliability and validity measures may provide a model for institutions to implement similar assessment methods efficiently.

Evaluation of SOAP note performance in a pharmacotherapeutics course after implementation of a peer-review process.

INTRODUCTION: Subjective, objective, assessment, and plan (SOAP) notes are widely utilized within pharmacy curricula. Implementation of a peer-review process for these assignments may improve student competence. Evidence is lacking regarding correlation between student-to-student peer-review and faculty grades for SOAP notes and correlation of SOAP note peer-evaluation to individual performance. METHODS: Third year pharmacy students completed two SOAP notes in Pharmacotherapeutics IV. A peer-review process was implemented in spring 2020. After each SOAP note, students were assigned a peer's assignment to evaluate utilizing the same 50-point rubric as the faculty. SOAP note grades were compared between the 2019 and 2020 cohorts. RESULTS: Analysis included 98 students in spring 2019 and 92 students in 2020. SOAP note faculty grades were different between 2019 and 2020 for the first SOAP note (37.6 vs. 41.1, P < .001) but not for the second (42.3 vs. 42.7, P = .49). Peer-review grades were higher for both SOAP notes compared to faculty grades. Peer-review grades did not differ between the first and second SOAP notes (45.5 vs. 45.6) while faculty grades did (41.2 vs. 42.7). The difference in scores from peer-review compared to faculty grades was -4.4 points for the first SOAP note and - 2.9 points for the second SOAP note (P = .08). CONCLUSIONS: The peer-review process did not appear to improve SOAP note performance. Students tended to score better on the second note and appeared to gain proficiency in the peer evaluation process, suggesting a possible benefit of including additional SOAP notes.

Evaluating the quality of multiple-choice questions in a NAPLEX preparation book.

INTRODUCTION: There is a plethora of preparatory books and guides available to help study for the North American Pharmacist Licensure Examination (NAPLEX). However, the quality of questions included has not been scrutinized. Our objective was to evaluate the quality of multiple-choice questions (MCQs) construction in a commonly used NAPLEX preparatory book. METHODS: Five students and two faculty members reviewed MCQs from the RxPrep 2018 edition course book. Item structure and utilization of case-based questions were evaluated using best practices for item construction. Frequency of item writing flaws (IWF) and utilization of cases for case-based questions was identified. RESULTS: A total of 298 questions were reviewed. Twenty-seven (9.1%) questions met all best practices for item construction. Flawed questions contained an average of 2.53 IWF per MCQ. The most commonly identified best practice violations were answer choices containing differing length and verb tense (21%) and question stems containing too little or too much information necessary to eliminate distractors (16.6%). Of the case-based questions, the majority (61.9%) did not require utilization of the provided case. CONCLUSIONS: This pilot analysis identified that a majority of MCQs in one NAPLEX preparatory source contained IWF. These results align with previous evaluations of test-banks in published books outside of pharmacy. Further evaluation of other preparatory materials, to expand on the findings from this pilot analysis, are needed to evaluate the pervasiveness of IWF in preparatory materials and the effect of flawed questions on utility of study materials.

Effectiveness of a pharmacy teaching certificate program offered to practicing pharmacists.

INTRODUCTION: Regardless of the practice setting, pharmacists' roles often expand beyond patient-centered services and into health-related educational roles for patients, healthcare providers, and student pharmacists. Teaching roles of a clinical pharmacist can include providing invited presentations (such as continuing education lectures), teaching colleagues about new clinical initiatives, precepting students during advanced pharmacy practice experiences (APPEs) and introductory pharmacy practice experiences (IPPEs), and precepting residents on rotations. This program was designed to evaluate the effect of a university-affiliated pharmacy teaching certificate program on participant knowledge and perceived confidence in specific areas of academic pharmacy and preceptor development. METHODS: A 10-month program consisting of four live modules with associated continuing education (CE) credit was offered to pharmacy residents and practicing pharmacists in the surrounding area. The program format utilized didactic and active learning strategies with "at home" assignments to reinforce material presented. Content knowledge was measured with written summative assessments provided at the start and end of each module. Perceived confidence was assessed through online surveys provided at the start and end of the program. All results were evaluated using paired t-tests. RESULTS: Data were obtained from 14 participants in the first cohort and 18 participants in the second cohort. For each of the four modules, there was a significant difference in pre- and post-knowledge assessments. Confidence data was also significantly increased regarding topics in each module. CONCLUSION: Knowledge assessments and perception surveys suggest that this university-affiliated teaching certificate program improved both knowledge and perceived confidence for program participants.

Predicting success in pharmaceutical calculations.

INTRODUCTION: Understanding preadmission predictors of success in pharmacy calculations, an important aspect of pharmacy practice, could impact admissions selection and advising practices. The objective of this study was to determine which student specific preadmission variables best predict success in a pharmaceutical calculations course. METHODS: Preadmission data from 388 students who completed a one semester pharmaceutical calculations course between 2013 and 2016 were evaluated. This retrospective analysis was performed to determine which of the independent preadmission variables were positively correlated to the pharmaceutical calculations course grade. Fifteen preadmission variables, including demographics, grade point average (GPA), Pharmacy College Admission Test (PCAT) score, prior degrees, and number of pre-professional credit hours completed, were evaluated. Descriptive statistics were used for baseline characteristics and relative importance analysis was used to examine association between the dependent and independent variables. RESULTS: The relative importance analysis revealed eight of the fifteen preadmission variables were significantly correlated with final grades in pharmaceutical calculations. Overall, 26.1% of the variance was explained by these variables. GPA (cumulative and science specific) and PCAT (quantitative and verbal) were the strongest indicators. CONCLUSIONS: Preadmission GPA and PCAT scores were the best predictors of success in a pharmaceutical calculations course. About one quarter of the variance was explained by the identified predictive variables, therefore other factors, not evaluated in this study, likely influence the success in pharmaceutical calculations and should be investigated to determine a more dependable set of predictors.

A web-based review game as a measure of overall course knowledge in pharmacotherapeutics.

BACKGROUND AND PURPOSE: Implementation of games is common in pharmacy education, but limited information exists about the relationship between students' game performances and course grades. Our study was designed to determine if scores on a comprehensive web-based review game correlate to overall pharmacotherapeutics course and course series grades. EDUCATIONAL ACTIVITY AND SETTING: Two cohorts of students in Pharmacotherapeutics IV, the last course in a four course series, were administered a web-based quiz review game (Kahoot!) at the end of the course. Student performance on the game was compared to grades throughout the Pharmacotherapeutics course sequence to determine if the games were reflective of the students' performances. All students enrolled in Pharmacotherapeutics IV in 2016 and 2017 were included in the analysis. Pearson correlation was performed on the scores from the review game compared to the grades in the pharmacotherapeutics course series. FINDINGS: A total of 197 students, 111 in the 2016 cohort and 86 in the 2017 cohort, were included in the analysis. The correlation coefficient (r) for the review scores and Pharmacotherapeutics IV course grade was 0.399 and 0.461 for the 2 cohorts (p < 0.001). Almost all component comparisons between the review scores and the other pharmacotherapeutics course grades were also significantly correlated (p < 0.01). SUMMARY: Student performance on the review game significantly correlated with pharmacotherapeutics course grades. Review games are fun tools to review course content and can serve as an effective method to determine student understanding, progression, and knowledge.

How-to-guide for writing multiple choice questions for the pharmacy instructor.

BACKGROUND: Writing multiple choice questions (MCQ) takes a lot of practice. Often, pharmacy practitioners lack the training to write effective MCQ. Sources for instruction in effective MCQ writing can be overwhelming with numerous suggestions of what should and should not be done. PURPOSE: The following guide is prepared to serve as a succinct reference for creation and revision of MCQ by both novice and seasoned pharmacy faculty practitioners. METHODS: The literature is summarized into 12 best practices for writing effective MCQ. Pharmacy specific examples that demonstrate violations of best practices and how they can be corrected are provided. IMPLICATIONS: The guide can serve as a primer to write new MCQ, as a reference to revise previously created questions, or as a guide to peer review of MCQ.

Student perceptions of group examinations as a method of exam review in pharmacotherapeutics.

OBJECTIVE: Evaluate student perceptions of learning outcomes, group dynamics, and team-based skill development during group-retake exams in pharmacy education. DESIGN: Group-retake examinations utilizing distributed practice were implemented in a pharmacotherapeutics course to serve as a post-individual exam review. A survey was designed to assess student perceptions of these group-retake exams. ASSESSMENT: Students perceived group-retake examinations to be beneficial in respect to retention and application of content. Peer teaching was reported to be effective and occur frequently. Minimal hostility and stress were reported. Consensus answers were reached often and this improved throughout the semester. Overall, 92% of students agreed that group-retake examinations should be continued. CONCLUSION: Group-retake examinations were perceived by students to be an effective method for exam review within pharmacy education.